gtfs cut vertices
Ido Klein
28/12/2021
packages and definations
piping and data manipulation - {tidyverse} geometry manipulation - {sf}, {lwgeom}, {nngeo} network analysis - {sfnetworks}, {tidygraph} gtfs loading - {gtfstools} visualization - {mapview} hebrew encoding
library(tidyverse)
library(sf)
library(lwgeom)
library(nngeo)
library(sfnetworks)
library(tidygraph)
library(gtfstools)
library(mapview)
Sys.setlocale(locale = "hebrew")## [1] "LC_COLLATE=Hebrew_Israel.1255;LC_CTYPE=Hebrew_Israel.1255;LC_MONETARY=Hebrew_Israel.1255;LC_NUMERIC=C;LC_TIME=Hebrew_Israel.1255"loading gtfs from 7.7.2021 (takes a while)
The {gtfstools} package allows for loading of an entire gtfs file. it takes a while but it is worthwhile because it is more clean.
in this stage, we perform the analysis on beersheva only
spo_gtfs <- read_gtfs("D:/Downloads/gtfs.zip",encoding = "UTF-8")## Warning in data.table::fread(file.path(tmpdir, file_txt), select =
## fields_classes, : Found and resolved improper quoting out-of-sample. First
## healed line 50088: <<׳׳¨׳›׳– ׳©׳™׳§׳•׳ "׳׳™׳׳"/׳”׳׳“׳¢,EN,"Ilan" Rehabilitation
## Center/Hamada>>. If the fields are not quoted (e.g. field separator does not
## appear within any field), try quote="" to avoid this warning.creating lines
using the routes, trips and shapes, we create a geometry for each line makat.
we transform the goemetry from wgs84 to ITM.
lines <- spo_gtfs$routes %>%
# at first, only for Dan Beer sheva
filter(agency_id == 32) %>%
select(route_id,route_desc) %>%
# join with trips
left_join(spo_gtfs$trips, by = "route_id") %>%
# get only distinct values
select(route_desc,shape_id) %>%
distinct() %>%
# join with shapes
left_join(spo_gtfs$shapes, by = "shape_id") %>%
# arrange points in each line by sequence
arrange(route_desc,shape_id,shape_pt_sequence) %>%
# create linestring for each line
group_by(route_desc,shape_id) %>%
nest() %>%
mutate(line = st_sfc(map(data, ~.x %>%
select(shape_pt_lon ,shape_pt_lat) %>%
as.matrix() %>%
st_linestring() ),crs=4326),
# transform to ITM
line = st_transform(line,2039)) %>%
select(-data) %>%
ungroup() %>%
st_sf()
m1 <- mapview(lines,zcol = "route_desc")
m1@map1. exploding all of the lines above to segments
2. subsetting to only distinct segments
3. parsing all distinct segments to a spatial network, that includes edges and nodes
the nodes are one source for intersections, however some of them are irrelevant (see stage 5) 4. adding intersections that are created by intersection of two segments without a connecting node.
this is somewhat problematic because it assumes we deal with a planar graph, and that means that bridges and tunnels will subdived segments.
5. smoothing segments - removing nodes that are connected to only 2 edges, making more uniform edges.
this eliminates nodes that were created in the lines explosion and are irrelevant for the analysis.
intersections <-
# explode all lines
st_segments(lines$line) %>%
st_as_sf() %>%
# get only distinct segments
distinct() %>%
# parse as sfnetoworks object
as_sfnetwork() %>%
# subdivide segments that intersect (assuming planar graph)
convert(to_spatial_subdivision) %>%
# smooth useless nodes
convert(to_spatial_smooth) %>%
st_as_sf() ##
|
| | 0%
|
|= | 1%
|
|== | 3%
|
|=== | 4%
|
|==== | 6%
|
|===== | 7%
|
|====== | 8%
|
|======= | 10%
|
|======== | 11%
|
|========= | 13%
|
|========== | 14%
|
|=========== | 15%
|
|============ | 17%
|
|============= | 18%
|
|============== | 20%
|
|=============== | 21%
|
|================ | 23%
|
|================= | 24%
|
|================== | 25%
|
|=================== | 27%
|
|==================== | 28%
|
|===================== | 30%
|
|====================== | 31%
|
|======================= | 32%
|
|======================== | 34%
|
|========================= | 35%
|
|========================== | 37%
|
|=========================== | 38%
|
|============================ | 39%
|
|============================= | 41%
|
|============================== | 42%
|
|=============================== | 44%
|
|================================ | 45%
|
|================================= | 46%
|
|================================== | 48%
|
|=================================== | 49%
|
|=================================== | 51%
|
|==================================== | 52%
|
|===================================== | 54%
|
|====================================== | 55%
|
|======================================= | 56%
|
|======================================== | 58%
|
|========================================= | 59%
|
|========================================== | 61%
|
|=========================================== | 62%
|
|============================================ | 63%
|
|============================================= | 65%
|
|============================================== | 66%
|
|=============================================== | 68%
|
|================================================ | 69%
|
|================================================= | 70%
|
|================================================== | 72%
|
|=================================================== | 73%
|
|==================================================== | 75%
|
|===================================================== | 76%
|
|====================================================== | 77%
|
|======================================================= | 79%
|
|======================================================== | 80%
|
|========================================================= | 82%
|
|========================================================== | 83%
|
|=========================================================== | 85%
|
|============================================================ | 86%
|
|============================================================= | 87%
|
|============================================================== | 89%
|
|=============================================================== | 90%
|
|================================================================ | 92%
|
|================================================================= | 93%
|
|================================================================== | 94%
|
|=================================================================== | 96%
|
|==================================================================== | 97%
|
|===================================================================== | 99%
|
|======================================================================| 100%m2 <- mapview(intersections)
m2@mapget relevant stops
using the routes, trips, stop_times and stops tables, we create a layer where each stop connected to its line makat and to its place in the stops sequence.
stops_routes <- spo_gtfs$routes %>%
# at first, only for Dan Beer sheva
filter(agency_id == 32) %>%
# join change until getting to stops
left_join(spo_gtfs$trips, by = "route_id") %>%
left_join(spo_gtfs$stop_times, by = "trip_id") %>%
left_join(spo_gtfs$stops,by = "stop_id") %>%
# get only distinct stops per line
select(route_id,route_desc,shape_id,stop_id,stop_sequence,stop_lat,stop_lon) %>%
distinct() %>%
st_as_sf(coords = c("stop_lon","stop_lat"),crs = 4326) %>%
st_transform(2039)
m3 <- mapview(stops_routes,zcol = "route_desc")
m3@mapcreate makats
get all line makats in order to iterate over the lines and stop_routes
makats <- lines$route_desccutting the line
using st_network_blend as the main workhorse
for each line makat: get line get stops along route transform line to spatial network blend stops along route (first order segments) to spatial network using 10 meter snapping tolarance blend other stops (first order segments) using 10 meter snapping tolarance blend intersections using 0.1 meter snapping tolerance in the map, we see the cuts in orange, the segments in light blue, the intersections in yellow and the stops in blue. you can turn the layers on and off to check the result.
line_with_all_cuts <- map(makats,function(x){
print(x)
line <- lines %>% filter(route_desc == x)
stops <- stops_routes %>% filter(route_desc == x)
line %>%
as_sfnetwork() %>%
st_network_blend(stops,10) %>%
st_network_blend(stops_routes,10) %>%
st_network_blend(intersections,0.1)
})## [1] "15112-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "15112-1-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "15112-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "15112-2-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "20034-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "20034-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "21025-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "21025-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "23024-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "23024-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "23032-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "23032-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "25023-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "25023-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "27022-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "27022-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "30009-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "30009-1-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "30009-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "30009-2-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "30017-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "30017-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "31018-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "31018-1-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "31018-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "31018-2-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "32016-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "32016-1-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "32016-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "32016-2-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "33013-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "33013-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "33014-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "33014-1-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "33014-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "33014-2-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "35007-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "35007-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "38008-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "38008-1-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "38008-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "38008-2-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "39006-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "39006-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "39012-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "39012-1-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "39012-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "39012-2-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "39081-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "39081-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "43080-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "43080-1-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "43080-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "43080-2-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "45005-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "45005-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "45027-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "45027-1-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "45027-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "45027-2-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "52004-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "52004-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "55003-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "55003-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "61002-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "61002-1-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "61002-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "61002-2-1"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "61082-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "67011-1-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries## [1] "67011-2-0"## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries
## Warning: st_network_blend assumes attributes are constant over geometries# vis of example
line_with_all_cuts[[3]] %>% activate(nodes) %>% st_as_sf() -> cuts
line_with_all_cuts[[3]] %>% activate(edges) %>% st_as_sf() -> liness
m4 <- mapview(cuts,col.regions = "orange")
m5 <- mapview(liness)
m6 <- mapview(intersections,col.regions = "yellow")
m7 <- mapview(stops_routes,col.regions = "blue")
m8 <- m5+m6+m7 + m4
m8@maplimitations
only beer sheva needs snapping parameter for type 2 segments doesnt account for directions in snapping type 2 segments ## next steps creating tables discussing joining different GTFSs